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ABSTRACT 

An authoring system is defined as a computer program 
which eliminates the need for course instructors to learn computer 
programming, but instead constructs pedagogical software based on 
instructors' responses to questions on teaching strategy and course 
material. This paper outlines an authoring system which not only 
exhibits this ease of use and flexibility in its design, but 
demonstrates intelligent behavior, one of the significant interactive 
qualities of computer assisted language learning. Intelligence is 
simulated by daemons (a program submodule, typically highly 
parameterized), each of which seeks a specific error in student 
input, providing the student with immediate error-contingent 
feedback. The user defines a daemon by providing a name and 
specifying the action the daemon will perform. The actions correspond 
to three decision points: string comparison, lexical search, and 
sentence search. The model, which allows the course designer to 
create any type of language exerci^se, is demonstrated in German as 
the target language, and English as the source language. An 
intelligent workbook is constructed through selecting, from a pool of 
exercises that has been created, those that will appear in the 
workbook, and assigning an order to them. The system described here 
was written in Allegro CommonLisp (TM) and runs on the Apple 
Macint osh (TM) . Contains five references. (Author/MAS) 
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The Intelligent Workbook 

T. HEIFT and P. MCFETRIDGE 
Department of Linguistics 
Simon Fraser University. Burnaby, B.C. Canada V5A IS6 
E-mail: heift@sfu.ca mcfet@cs.sfu.ca 

Abstract: Burghardt (1984) defines an authoring system as a computer program 
which eliminates the need for course instructors to learn computer programming but 
instead constnicLs pedagogical software based on the instructors responses to ques- 
tions on teaching strategy and course material. In this paper we will outline an 
authoring system which not only exhibits this ease of use and flexibility in its 
design, but demonstrates intelligent behavior, one of the significant interactive qual- 
ities of Computer-Assisted Language Learning. Intelligence is simulated by dae- 
mons each of which seeks a specific error in student input, providing the student 
with immediate error-contingent feedback. The model, which allows the course 
designer to create any type of language exercise is demonstrated with German as the 
target language and English as the source language. 
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Introduction 

All compelling arguments for Computer-Assisted Language Learning (CALL) assume intelligent responses 
to students" input. Without intelligence the computer is merely another medium for presenting information, one 
not especially preferable to a static medium such as print. In order to go beyond the multiple choice questions, 
relatively uninformative answer keys and gross mainstreaming of students characteristic of workbooks, a model 
should emulate significant aspects of a student-teacher interaction. 

This paper presents an authoring system for CALL. The system is designed to produce an intelligent tutor, 
rather th;in m electronic workbook: the principal advantage and contradistinction being that an intelligent tutor- 
ing system provides the student with evaluative or error-contingent feedback (Alessi and Trollip, 1985). Venezky 
& Osin (1991) stress that "for almost all cognitive learning, instruction is enhanced by evaluative feedback. In 
many cases it is essential, if any learning is to occur. Translation of a foreign language is a prime example of the 
latter situation (pg. 9)." 

Turning to a concrete example of error-contingent feedback, consider the following grammar exercise 
intended to practice word order in subordinate clauses in German. In the main clause the verb occurs in second 
position: however in a subordinate clause the verb must be in sentence-final position. The student is presented 
with the exercise: 

(1) Build a sentence with the words provided- 

Ich / nehmen / zwei / Mentis / well / ich / haben / Hunger. 

The simplest system merely reports an error if the student does not supply thr. correct answer, "Ich nehme 
zwei Menus, weil ich Hunger habe", perhaps revealing the correct answer after a number of tries. By compari- 
son, error-contingent feedback responds with a description of the error, and performs a deeper linguistic an;ilysis 
in order to isolate the source of the error. If the student responds with "Ich nehme zwei Menus, weil ich *habe 
Hunger, the system responds with "incorrect word order in subordinate clause". The more detailed error analysis 
guides the student toward the correct answer, and provides an invaluable record of the student's performance for 
later evaluation and remedial work. 

The error-contingent feedback which serves as the basis for effective teaching in the language classroom Ccin 
be transferred to the computer. This has been previously demonstrated with ;m intelligent tutoring system on a 
small micro computer platfonn teaching subordinate clauses in German (Heift 1993 Heift & McFelridgc 1993) 
In this project, all error checking routines - which we have called daemons - were built rather piiinsUi^ungly 
inis process is time consuming and requires expert knowledge of programming ;is well as second language 
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instruction. The project we report here is an authoring system for creating intelligent workbooks. It provides a 
method for specifying which daemons are required for each exercise and how they are parameterized. The com- 
puter code wliich analyses student errors is automatically generated. 

The Daemon Approach 

Error-contingent feedback is provided through the use of daemons which only seek errors relevant to the 
exercise. A daemon is a program submodule» typically highly parameterized^ which seeks a particular error and 
lakes remedial action when that error is discovered. Daemons are conceptually simple but can simulate intelli- 
gence. They achieve this» in part» because the pedagogical principles of 'selection* and 'gradation' constrain the 
problem domain — the range of possible errors in any given exercise is small, or at least finite, compared to many 
natural language processing tasks. They are simple in the sense that each daemon is responsible for checking 
only one type of error — in our example above, for instance, one daemon would test solely for word order, while 
separate daemons would each check verb inflection, spelling, etc. This modularity allows the programmer to cre- 
ate a pool of daemons for each exercise, adding daemons from previous exercises to the current one. The daemon 
approach ensures relevance of response — the order in which daemons are activated keeps the point of the cur- 
rent exercise salient. Additionally, the overall system can easily be extended to encompass new phenomena by 
adding new daemons to the pool. 

The daemon approach also allows us to make efficient use of a small computer platform, which in turn 
improves the student/program interaction. Generally, the amount of knowledge and processing time required by 
a tutoring program increases with the sophistication of the error analysis. We can, however, establish an approxi- 
mate hierarchy which reflects the kinds of errors most likely to occur, and call the daemons in that order. The 
hierarchy itself is based on what has been taught most recently, and to some extent, on a contrastive analysis 
between the native/target language of the student. An additional consideration might be the complexity of the 
grammatical construction. We can reasonably assume that given two constructs of different complexity, the more 
difficult of the two will result in more frequent and more persistent errors. 

Conceptually powerful and computationally efficient, the daemon approach offers still Jinother attraction: 
reusability. In the same way that daemons can be conjoined and brought to bear on a series of exercises, much of 
the research and development time spent in creating one tutoring program can be applied to the next. The ratio- 
nale underlying an intelligent tutor*s design then provides the framework for a higher-level authoring system. 

Our goal is to produce tutoring systems which provide intelligent and infonnative feedback to students' 
errors. Ours is a more ambitious project than many on two parameters. First, we are attempting to open up the 
domain of errors which the system can analyze. The simplest system associates messages with pjirticuhir 
responses and constrains the student to select one. We wish to allow students freedom in their responses Jind con- 
sequently require greater generality in the routines which analyze the students's input. Second, the system we 
present here is not a tutoring system itself; it is a workbench for creating a tutoring system. We have decomposed 
the process of error detection into a small set of subroutines which the program writer can use to create daemons 
Jind to build exercises. The resulting tutoring system is one we call an intelligent workbook because it has the 
superficial appearance of an electronic workbook, but incorporates sophisticated error iinalysis and feedback. 

We illustrate how a set of daemons is constructed by first considering the problem of checking that the verb 
in the example above is inflected correctly and is in the correct position. The portion of a flow chai t for i\r\ exer- 
cise relevant to this task is given in figure 1. Each decision point in the flow chiirt corresponds to a daemon. The 
first daemon checks that the final word in the sentence is the correctly inflected verb. It functions jis a gateway to 
other daemons: if the last word in the sentence is not the expected word, the flow of control is pjissed to daemons 
which determine what error has been made. The first of these scans the sentence for the verb and if successful 
displays an error message informing the student that the verb is not in the correct position. It tenninates the pro- 
gram to give the student an opportunity to correct the mistake. The second daemon scjins the dictionjiry entry of 
ihe verb for i\n inflected form corresponding to that which the student entered. If it finds one, then the verb musl 
bo incorrectly inflected and an error message reporting this is displayed. Finjilly, if the student's error is so severe 
that neither daemon can successfully amilyze it, a generic error message is displayed. 
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Figure 1 
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A System Overview 

The task of the program writer is to convert the flow chan in figure 1 into a program. The system we 
dcscnbe here IS a workbench which modularizes the construction of a program and presents the writer with a set 
of packaged daemons. Creaung an intelligent workbook is a process of several well-defined stages. 

In the first stage the set of daemons that will be required is defined. We describe this process in the section 

L h V? ""'"t^ P'^""'' ^ cumulative, and consequently have provided edi- 

tors which allow the program ^.Tlter to amend the definitions of daemons. It is also possible to add new daemons 
to the pool at any time. As a consequence, it is possible to begin with a workbook which can respond to only the 
fnTirovS'st^ct^""'' functionality into the workbook without chang- 

After the pool has been defined, the writer defines the exercises which comprise the workbook As in the 

exercise is the set of daemons which are assigned to the exercise. When a daemon is assigned to an exercise its 
paran^cers must also be assigned. For example, the task given to a student in example (1) above re^ulesTaUhe 
^erb h'^ two T^""- '''''' '''' lastUd\nte rden" tswe 

the w^rlJl'[-'^' """'^^^^^ '^''^"^"^ ""^''^ ^'^^^^i^^^ ^ to wear ^md saving 

ho workbook. The workbook is a stand alone application which presents the student with a series of cxe ci as 
and applies the daemons to the students responses to each exercise. exercises 

Daemon Functionality 

formI^,n3?;i '^"^r"" ^''"""-"^""^"^'^ ^hree kinds of error .'analyses. The first decision point compares two 

a™ 0 ;e;e er;t1 'iTT '"'""h"' "^'"P^^^"' ^^^""'^ ^^-^ point rclTcs on 
me sentence. If the verb c:in be found somewhere in the sentence, the .'.ssociated error mess<'ige is dis- 
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played. The final decision point searches the lexicon for the form which the student entered. To this point of our 
analysis of errov detection, this list exhausts the possibilities. 

As an example, the definition of the daemon corresponding to the second decision point is illustrated in fig- 
ure 2. 

Figure 2 
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The user defines a daemon by providing a name and specifying the action which the daemon will perfonii. 
The actions correspond to three decision points in the example in figure 1: 

(1) String Comparison 

In a substring search daemons are directed to specific positions within the sentence. This applies to daemons 
which check word position or verb inflection, as in example (1). Further examples are daemons which check 
prepositions, inflection of attributive adjectives, gender, case, etc. 

(2) Lexical Search 

In a lexical search the daemons refer :o a database, such as a glossary, or dictionary. For example, in the flow 
chart above the third decision point corresponds to an attempt to find the last word in the sentence in the dictio- 
nary under the lexical entry of haben. If it is found, then the student has placed the verb in the correct position 
but has inflected it incorrectly. 

(3) Sentence Search 

This kind of daemon searches the entire sentence for specific strings; for example, if a verb is not found in 
the correct position, scanning the entire sentence for the verb will establish whether the student has placed the 
verb incorrectly. The example in figure 2 illustrates the construction of a daemon of this t>pe. As the flow chiirt in 
figure 1 illustrates, this daemon is activated only if it is previously established that the last string in the sentence 
is not a correctly inflected form of the verb. The ScanPorVerb daemon will look through the sentence for the verb 
and display an error message if it successfully find the verb. 

The type distinction is equivalent to the yes/no paths in the flow chart. A positive type will display an associ- 
ated message if the specified action is successful. A negative type will display an associated message if the spec- 
ified action is unsuccessful. In the flow chart in figure 1 , the daemon corresponding to the first decision point will 
be negative. Its function is not to display a message but to serve as a gateway to the other daemons. 

Two other properties of a daemon are determined by the state of the check boxes in the lower left. If the icr- 
minate box is checked, all further processing is stopped after the message is displayed. This feature gives the stu- 
dent an opportunity to correct the error before presentation of other errors. By default, all daemons which display 
a message terminate. 

- The user model check box establishes a count for the error tracked by the daemon. Each time a daemon suc- 
cessfully discovers an error, the count associated with it is incremented. This count can be displayed at the end of 
a session or be used to direct the student to remedial exercises. 

Interrelation of Daemons 

It is often necessary for daemons to cooperate in the analysis of a student's input. It is also useful to be able 
to refer to a large group of daemons by a single name; this is particularly true if the same set of daemons is used 
in a number of exercises. We have provided two different ways of combining daemons into larger prognun struc- 
tures. 
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Groups of daemons which are ordered but not mutually exclusive can be built up similarly. 
Defining an Exercise 

The penultimate step in creating a workbook is setting the exercises. We provide a window in which the text 

VcrbDacmon defined in figure 3, is assigned to this exercise. At this point the system requests values for the 
parameters of each of the subdaemons. Hie ChedkVerb daemon will be parame JzS w th ^d 8 "^at s 
Jhe answer is inco^ect if the S'^ word is not Habe. Th. ScanForVerb daemon wiU bVpa^ltrizTd witJ^^ ' 
T^is daemon wi 1 scan the student's input for this exercise and signal the associated enTn^^ssageT^t s 
found. Fma^ly, the LooklnLexicon daemon will be parameterized with haben and 8. It will sS the txfc^ 
entry of ,aben for the word found in 8^ position of the input and signal the associated em^T^eLge If it fs 

Figure 4 
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Building an Intelligent Workbook 

The final step in the process of creating a workbook is selecting from the pool of exercises that has been cre- 
ated, those which will appear in the workbook and assigning an order to them. Once this has been done, the 
workbook is saved as a stand alone application. The workbook displayed to the student consists of a series of 
windows in which the exercises are presented and the student's input is analysed. An example of a window is 
given in figure 5. 

Figures 
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Conclusion 

The work described here is research in progress. We have initially approached the problem of error an^dysis 
as one of anticipating possible errors. We believe that this approach has considerable potential, particularly for 
exercises where students' errors are easily targeted. We have demonstrated that a tutoring system of considerable 
sophistication can be built from primitive building blocks. 

We intend to add further functionality to the lists of actions which a daemon may take Jind pJiriicukiriy to 
include natural language processing as part of the error analysis. Natural language processing is p^irticularly 
good at analyzing grammatical sentences, but has proved difficult to use in error anidysis. Parsers normally fml 
numerous times in the analysis of a sentence and this namral failure is difficult to sepiiraie from real errors. Our 
initial investigations suggest that by assigning natural language processing to a daewion, it is possible to con- 
strain the search space so that the natural language processing system is attempting to solve a parlicuhir problem 
and its success or failure is relevant to the daemon. Our first efforts will include morphological an^iiysis so that 
students' treatment of inflections can be more deeply analysed. The system described here was written in Allegro 
CommonLisp™ and runs on the Apple Macintosh™. 
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